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Abstract — We investigate the sctieduling of a common resource 
between several concurrent users wlien tlie feasible transmission 
rate of each user varies randomly over time. Time is slotted 
and users arrive and depart upon service completion. This 
may model for example the flow-level behavior of end-users 
in a narrowband HDR wireless channel (CDMA IxEV-DO). As 
performance criteria we consider the stability of the system and 
the mean delay experienced by the users. Given the complexity 
of the problem we investigate the fluid-scaled system, which 
allows to obtain important results and insights for the original 
system: (1) We characterize for a large class of scheduling 
policies the stability conditions and identify a set of maximum 
stable policies, giving in each time slot preference to users being 
in their best possible channel condition. We find in particular 
that many opportunistic scheduling policies like Score-Based 
[8], Proportionally Best [1] or Potential Improvement [4] are 
stable under the maximum stability conditions, whereas the 
opportunistic scheduler Relative-Best [9] or the c/i-rule are not. 
(2) We show that choosing the right tie-breaking rule is crucial 
for the performance (e.g. average delay) as perceived by a user. 
We prove that a policy is asymptotically optimal if it is maximum 
stable and the tie-breaking rule gives priority to the user with 
the highest departure probability. We will refer to such tie- 
breaking rule as myopic. (3) We derive the growth rates of the 
number of users in the system in overload settings under various 
policies, which give additional insights on the performance. (4) 
We conclude that simple priority-index policies with the myopic 
tie-breaking rule, are stable and asymptotically optimal. All our 
findings are validated with extensive numerical experiments. 



I. Introduction 

Next generation wireless networks are expected to support 
a wide variety of data services. Due to fading and interference 
effects, for each user, the quality of a downlink channel, and 
hence its transmission rate, fluctuates over time. This has 
triggered a large amount of work aiming at understanding the 
performance of channel-aware scheduling policies. It is by now 
accepted that so-called "opportunistic schedulers" have many 
deskable properties (see for example [9]). A policy is called 
opportunistic if it takes advantage of the channel fluctuations 
by serving a user whose channel condition is in a good state 
with respect to its own statistical behavior With the objective 
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of minimizing mean users' delay, there arises a key tradeoff 
in the design of scheduling mechanisms between making full 
use of the opportunistic gains, and prioritizing users having 
small residual service sizes. 

Broadly speaking, researchers have explored scheduling in 
wireless systems both at the packet level and at the flow level. 
In packet-level models it is typically assumed that there exists 
a finite number of permanent users. The focus of the scheduler 
is on the number of packets in the queue of each user. We 
refer for example to [34], [2], [33], [17], [3], [25], [29] for 
this line of research. In a flow-level model instead, users arrive 
randomly to the system and leave after receiving their finite- 
sized service demands. This allows to capture the performance 
as perceived by the end-users, see for example [8], [21], [9], 
[26], [22], [1], [4], [30]. For surveys on flow-level modeling 
we refer to [24] and [10]. In [23], hybrid models are studied. 

The performance evaluation and optimization of wireless 
networks at the flow level has proved to be extremely chal- 
lenging. One of the most successful approaches has been the 
so-called time-scale separation argument (see [9], [11], [12], 
[29], [1], [30]) where it is assumed that at the flow scale the 
dynamics of the channel fluctuations can be averaged out. Un- 
der this time-scale assumption, it was shown in [11] that any 
utility-based scheduling policy is stable in a flow-level model. 
The authors of [1] make the same assumption when they 
discuss rate-stability for priority-index policies. In the context 
of optimal control, in [29], [30] scheduling mechanisms were 
introduced and evaluated. In [4] optimal control is studied 
without the time-scale separation assumption. The Lagrangian- 
relaxation method allowed the authors of [4] to construct the 
Potential Improvement (PI) scheduling policy, which is opti- 
mal for a relaxed optimization problem. In addition, several 
other policies have been proposed and numerically investigated 
in the literature, among others the Proportional Fair [14] 
discipline, the Score-Based (SB) algorithm [8], the Relative 
Best (RB) scheduler [6] and Proportionally Best (PB) [1]. 

To sum up, without a time-scaling separation argument, 
which is a rather strong assumption, the performance of 
opportunistic schedulers, regarding stability and performance 
perceived by the users, is not well understood. In order to gain 
better insight into the latter issue, in this paper we will study a 
flow-level model without the time-scale separation assumption. 
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More precisely, we assume that data users arrive randomly in 
time and have a finite amount of data to download. Time is 
slotted and the quality of the channel condition of each user 
varies per time slot. In every time slot at most one user may be 
served. We are interested in stability and optimization of the 
system. Given the complexity of the problem we first prove 
convergence of the fluid-scaled system towards a unique fluid 
limit. We note that the precise characterization of the fluid limit 
involves averaging phenomena of the scaled system which is 
not grasped by the usual description of weak fluid limits. 

The fluid-limit description allows us to obtain several im- 
portant results and insights for the original wireless system. 
First of all, we characterize the maximum stability conditions 
(the weakest possible conditions on the traffic parameters such 
that there exists a scheduling policy that makes the system 
positive recurrent) and show that the set of policies that are 
stable under the maximum stability condition have a very 
simple characterization: whenever there are users present that 
are currently in their best channel condition, only such users 
are served. These policies will be referred to as Best-Rate 
(BR) policies. Such a characterization was previously given for 
rate stability [1], but, to the best of our knowledge, stochastic 
stability was still an open issue. 

Second, for a large class of scheduling policies we deter- 
mine the exact stability conditions and conclude that many 
known opportunistic scheduling policies like SB, PB, or PI 
are stable under the maximum stability conditions, whereas 
the opportunistic scheduler RB or cfi-mle are not. 

Third, we demonstrate the importance for the choice of the 
tie-breaking rule when the goal is to optimize the performance. 
Until now, the literature proposed to break ties at random, 
see for example [6], [8], [9], [1]. We instead propose to 
give priority to the user with highest instantaneous departure 
probability when there are multiple users in their best channel 
conditions, which we refer to as the myopic tie-breaking 
rule. We prove that BR policies with the myopic tie-breaking 
rule are asymptotically fluid optimal and our numerical ex- 
periments further illustrate that the myopic tie-breaking rule 
significantly improves the performance. This in turn shows that 
simple priority-index policies that balance opportunistic gains 
with size-based information, will be both maximum stable and 
asymptotically optimal. 

Fourth, our convergence result allows to compare the per- 
formance of the various policies in an overload setting. More 
precisely, we determine the growth rates of the number of 
users in the various classes and find that BR policies with a 
myopic tie-breaking rule minimize the total growth rate. 

The paper is organized as follows. In Section II we present 
the model. In Section III we introduce the scheduling policies 
of interest and define their tie-breaking rules. In Section IV 
we derive fluid limits for a large class of policies. This allows 
to obtain our stability results as presented in Section V. In 
Section VI we characterize asymptotically optimal policies, 
both in normal regime and in overload, and discuss the 
importance of the tie-breaking rule. In Section VII we perform 
numerical experiments to validate our theoretical findings. 



II. Model Description 

We consider a time-slotted system serving one user in each 
time slot. This models for instance a CDMA IxEV-DO system 
as explained in Remark 1. There are K classes of users, and 
in each time slot the number of class-fc users arriving to the 
system, Ak, follows an i.i.d. sequence of random variables, 
with E{Ak) — Afe and E(A|) < oo. For each user the depar- 
ture probability varies over time as the quality of the channel 
is changing from slot to slot. The quality of the channel (or 
state of the channel) for a class-A; user is modeled as an i.i.d. 
sequence of random variables taking values in the finite set 
Afk ■= {1,2,..., iVfc}. For each time slot we let qk,,i denote 
the probability that a class-A; user is in channel state n £ JVk- 
Associated with channel state n is a departure probability fik,n- 
This can be used for instance to model a system in which the 
service requirements are geometric (see Remark 1). Without 
loss of generality we assume that the channel conditions are 
ordered such that < iik,i < /^fc,2 < • ■ ■ < Mfc.iVfe < 1, ™d 
<lk.NktJ'k,Nk 7^ 0, ^ fc- The channel condition of a class-fc user 
is independent of the channel conditions of all the other users 
and of the channel quality history. 

In each time slot t, a scheduler/policy / decides which user 
is served. Because of the Markov property of the system, we 
focus on policies that base decisions on the current number of 
users present in the various classes and on their current channel 
states. For a given scheduling policy /, let xl{t) denote the 
number of class-fc users in the various classes at time slot t and 
(t) = {X({t), . . . ,x{^{t)). Since the channel conditions 
are i.i.d. and independent of the process Xf{-), the process 
Xf is Markov and, in addition, for the modeling it is sufficient 
to focus on the Markovian description in terms of the number 
of users in each class, X^{-), instead of the number of users 
in each channel state. 

Let us introduce some more notation. We denote by |x| the 
li norm of a vector x. The notation x < y is used for the 
coordinate-wise ordering: Xi < yi,Vi. Finally, we denote by 
u.o.c. the uniform convergence on compact sets. 

Performance criteria: Our performance criteria are stability 
and long-run average number of users. We use the following 
definition for stability: 

Definition 1. A scheduling policy f is stable if the process 
xf is positive recurrent. 

Because of the time-varying channel conditions the system 
is not work-conserving, and hence it depends strongly on the 
employed scheduling policy whether the system can be made 
stable. We define the maximum stability conditions as the 
conditions on the traffic inputs such that there exists a policy 
that can make the system stable. A maximum stable policy is 
a policy that is stable under the maximum stability conditions. 
From the performance point of view it is therefore of crucial 
importance to design a scheduler that is maximum stable. 

Besides stability, another important performance measure is 
the long-run time-average holding cost, 

K T 

limsup-^^c,.E(X/(0), (1) 
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Fig. 1. Classification of schedulers. 



with Cfe > the holding cost incurred per time slot for having a 
class-fc user in the system. When = 1, V fc, this is equivalent 
to minimizing the mean sojourn time (cf. Little's law). 

Remark 1 (Modeling of a wireless data network). Our 

model, even though simple, captures some of the key properties 
of wireless communication systems. Time is slotted, as is the 
case in the CDMA IxEV-DO [5] and the OFDM-based LTE 
systems [31 ]. The available transmission rate of each user 
fluctuates due to fading effects, and as a consequence, it varies 
from one slot to another We note that in real systems the 
number of feasible transmission rates is finite (see [5]). 

One may classify the users into different classes based on 
their applications or traffic conditions for example. Let the 
service requirement (in bits) of a class-k user be a geometric 
random variable denoted by Bk, and let E(i3fe) denote its 
expectation. Let A denote the amount of bits transferred 
in one slot under the current channel condition. Note that 
in practice A will vary from slot to slot depending on the 
channel condition, and the allocation. The probability that 
a user leaves the system is approximately P(6 < Bk < 
6 + A|i3fc > 6) « /S./V,{Bk), which does not depend on 
the attained service b ( memoryless property of the geometric 
distribution). This expression becomes asymptotically exact as 
the ratio A/E(i?fe) goes to 0. Hence, this is the case if the 
mean service requirement (in bits) of a user is very large 
compared to the amount of bits that can be served in one 
slot. Let Sk,n denote the transmission rate (in bits per second) 
of a class-k user when the channel state is n. For the CDMA 
IxEV-DO system, the amount of bits transferred in one slot is 
^ — Sk,n ■ tc- Hence, the departure probability of a class-k 
user under channel condition n can be approximated by 

where is the length of the slot (for example tc = 1.67tos in 
the CDMA IxEV-DO system). In Section VII we perform nu- 
merical experiments with departure probabilities as obtained 
from a practical setting using (2). 

Remark 2 (Modeling an OFDM system). A natural exten- 
sion to our modeling framework will be to allow that in every 
slot multiple users can be served in parallel as it happens to 
be in the OFDM-based (3GPP LTE) system. In such a system 
there are M subcarriers and a subcarrier can be assigned 



at most to one user at a time. As reported in the literature, 
in an OFDM system a user can experience a deep fading in 
one subcarrier, while on the same subcarrier another user 
could be in good condition [32]. Under i.i.d. assumptions it is 
expected that for the metrics under consideration in this paper 
it is of no interest to serve multiple classes in parallel, that is, 
in every slot all the sub-carriers will be assigned to only one 
of the classes. However, serving multiple classes in parallel 
could be though of interest for other metrics like fairness, an 
issue that even though not studied here, definitely deserves a 
thorough investigation. 

III. Policies 

In this section we introduce scheduling policies that will 
be used throughout the paper Most of these policies are 
opportunistic (with the exception of the c//-rule), meaning that 
they take advantage of channel fluctuations by serving a user 
whose channel condition is currently in a good state, in some 
sense, with respect to its own statistical behavior 

We first introduce priority-index policies, which are very 
popular due to their simplicity from an implementation point 
of view. A priority-index policy is characterized by an index 
function that assigns an index to each user based solely on its 
class and its current state. 

Definition 2 (Priority-index policy). In every time slot, a user 
that has the highest index among all users present is served. 

Priority-index policies might need to be augmented with a 
suitable tie-breaking rule. Such a rule refers to the strategy 
adopted when there is a tie on the highest index value. A tie 
means that there are several users present having the highest 
index value, but these users belong to different classes. In the 
literature, most of the papers specify to break ties at random 
(see for example [8], [6], [I]). We define the myopic tie- 
breaking rule as the rule that among the users with the highest 
index, it selects the one with highest value for Cfc/i/cjv^, V k 
(the Cfc's refer to the holding cost introduced in Section II.) 
One of our main contributions will be that the choice for the 
tie-breaking rule is crucial for the performance of the system 
and that the myopic tie-breaking rule is close to optimal (this 
will be further developed in Sections VI and Section VII). 

In [8] the Score-Based (SB) policy is introduced. SB is 
a priority-index policy where the index value of a class-/c 
user in state n is given by 'Yl^=i^k,fn and ties are broken 
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at random. In [4] the Potential Improvement (PI) policy is 
introduced. PI is a priority-index policy with as index value 
I S '7fe,fi(Mfe,fi ^Mfe.ri), and the tie-breaking rule is the 

ri> n 

myopic tie-breaking rule. An important subset of the priority- 
index policies are the so-called weight-based policies. 

Deflnition 3 (Weight-based policy). A priority -index policy 
with index function cJkfJ-k.n- Here ujk denotes a class dependent 
weight 

Important examples of weight-based policies are: the c/i- 
rule (wfe = Cfe, with the holding cost), Relative Best (RB) 
[6] (oJk = ^/Y.n=i1k,nl^k,n), and Proportionally Best (PB) 
[1] (oJk — i/fJ'k,Nk)- For all these policies, ties are broken at 
random. 

It will be convenient to define the following two classes of 
policies, which play an important role in the results on the 
stability analysis and asymptotic optimality. 

Deflnition 4 (Best Rate (BR) policies). The BR policies are 
such that whenever there are users present that are currently 
in their best channel condition, i.e., in state N^, such a user 
is served. 

Deflnition 5 (Best Rate Priority (BRP) policies). The BRP 

policies are BR policies with a myopic tie-breaking rule. 

As a consequence of our main results, we will obtain that 
the classes of policies BR and BRP have desirable properties: 
In Section V we prove that any BR policy is stable under the 
maximum stability conditions and in Section VI we derive that 
BRP policies are asymptotically optimal. 

In Figure 1 we have summarized the various (classes of) 
policies. Note that SB, PB and PI are BR policies. This 
follows since the highest possible index value is 1 for SB 
and PB, and oo for PI, and these indices can only be obtained 
whenever a user is in its best possible channel condition. RB 
and the traditional c/i rule (i.e., giving in each time slot strict 
preemptive priority to the user having the highest Ck^ik,n) 
however do not belong to BR policies since, depending on 
the set of parameters, the index value of a class-fc user in 
state n ^ Nk might be larger than the index value of a class-Z 
user in state Ni. Remark also that PI is the only BRP policy. 

IV. Fluid limits and convergence 

In this section we study fluid-scaling limits for a large class 
of policies. Fluid scaling or time-space scalings, corresponding 
to "zooming out" the trajectories, have been used extensively 
to study stochastic processes with complex dynamics [16]. The 
limiting processes are usually much simpler to describe while 
they provide crucial insights on the behavior of the non-scaled 
version of the process. In particular, the convergence results 
will allow us to prove crucial results on stability and optimality 
of schedulers for the stochastic system. 

The fluid scaling consists in studying a sequence of systems 
indexed by r, i.e., for a given policy / we let x['^{t) denote 
the number of class-fc users at time t when the initial state 
equals X^(0) = rxkiO), k = l,...,K, with r e N, and 
Xf'''{t) = {X('''{t), ■ ■ ■ ,x{/'{t)). We are then interested in 



the fluid-scaled processes F/'''(t) := t > 0, k ^ 

1,...,K, with r'^(O) = a;(0). We can write 

Yi''{t) ^xm + -Yl Ms) - - E Sk,n{Tl::{rt)). o) 

s— 1 n— 1 

where Tl'l^{t) is defined as the cumulative amount of time 
that was spent on serving class fc in state n during the interval 
(0,t] and Sk,n{t) denotes the total number of class-fc users 
that have been completed while receiving service for a total 
duration of time t when being in state n. 

In order to derive stability and fluid optimality results, 
we will be interested in limits of the fluid-scaled process. 
In Section IV-A we will characterize a generic description 
of weak fluid limits (usually not unique) of Equation (3), 
following the same reasoning as in [15]. In Section IV-B we 
focus instead on a special class of policies for which we can 
prove convergence in probability towards a unique limit, which 
will be referred to as the strong fluid limit. (In [20] similar is 
done but only for a subset of the state space.) We discuss the 
differences between weak and strong fluid limits in more detail 
in Remark 5, after having introduced formally both concepts. 

A. Convergence towards weak fluid limits 

From (3), we obtain the following result that describes 
the generic characterization of weak fluid limits for a given 
policy /. The lemma will allow to determine maximum stable 
policies (Theorem V.2) and to characterize asymptotically 
optimal policies (Section IV). 

Lemma IV.l. For almost all sample paths w and any se- 
quence rk, there exists a subsequence rki such that for all 
fc = 1, 2, . . . , i^, n = 1, 2, . . . , iVfe, andt>0, 

Mm Y^'''''' {t) = yl{t), u.o.c, and (4) 

f oo 

lim = tI (t), U.O.C., 

i^oo rki 

with {yk{')j'''k ni')) continuous function. In addition, 

Viit) = Xk{0) +Xkt-J2 -"fc.n^nW. (5) 

ylit) > 0, t[JO) = a Ek.nrUt) < t, and t[ J-) are 
non-decreasing and Lipschitz continuous functions. 

Proof: The proof follows similarly to that of [13, Proposi- 
tion 4.12]. Define Ak{t) as the number of class-fc users that 
arrive in time slot t. We note that yl/j(2), . . . , are in- 

dependent and distributed according to Ak, with E(Afc) = A^. 
Since /ifc_„ is the probability of completing a class-fc user when 
it receives service while being at state n, by the law of large 
numbers, we obtain that, almost surely, 

1 1 

lim - Afc(s) = Afet, and lim -S'fc,„(ri) = /Xfc.„t. (6) 
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Since Tl'^{t) denotes the cumulative amount of time spent on 
serving class-fc users in state n in time interval [0, i], we get 

— ■ ■ < i — s, tor every t > s, 

r r 

i.e., Tl.'l^{t) := Tl''!^[rt)/r is Lipschitz continuous. Therefore, 
by the Arzela-Ascoli theorem [27] we obtain that, for almost 
every sample path w and for any subsequence r^, there exists 
a subsequence r/j, of such that Wthi^ooTI'I^'' (rt) /r = 
rl^it), u.o.c. Now, using Equations (3) and (6), it follows 
that lim,^oo i^/'*^'' (*) = with yf [t) as given in (5). □ 

We can now give our definition of weak fluid limits. 

Definition 6. We call the processes [t) and y^{t) (as 
obtained in Lemma IV. 1) weak fluid limits for policy f. 

Note that, in general, these fluid limits can be different 
depending on the sample path and the subsequence considered. 
A policy is said to have a unique fluid limit if, for all sample 
paths and all subsequences, the weak fluid limits coincide. 

B. Convergence towards a strong fluid limit 

In this subsection we will determine unique fluid limits for a 
special class of policies. More precisely, we will prove conver- 
gence in probability towards a unique limit. The derivation of 
the strong fluid limit will prove to be very useful: It allows to 
calculate the exact stabiUty conditions (policy dependent) (see 
Theorem V.l and the numerical Section VII). In addition, the 
exact characterization of the strong fluid limit provides crucial 
insights into the performance of the system, such as the growth 
rates of the number of users over time and monotonicity results 
with respect to the tie-breaking rule. 

Obtaining exact fluid-limit characterizations will require to 
deal with averaging phenomena: it may happen that one class 
of users reaches its stationary regime, i.e., is empty in the 
fluid scaling, before the other classes do. In this case, the drift 
of the other classes needs to be averaged with the stationary 
distribution of this class. Hence, a description of the fluid limit 
will involve averaged drifts as will be defined in (8) (we refer 
to this as second-vector fields, following [18]). 

We focus on the class of policies that induce partially 
increasing drift vector fields with uniform limits. In order to 
describe this class of policies we need first to introduce the 
drift functions and drift vector fields. For the stochastic process 
Xf{t) associated with a policy /, we define the drift function 
by df{x) := {6({x),...,d{^{x)), x € N^, with 

S{{x) ■.= Eix{{l)-x,\Xf{0)^x). 

(We will drop the superscript / when it is clear that we con- 
sider a unique policy.) We say that a vector field v : — > M.^ 
has uniform limits [12] if for any U C {1, • ■ . K}, there exists 
a function v'^ : NI'^I IR-'^ (constant when U = such that 

lim sup \v{x) — v^{xii)\ = 0, 

where xu denotes the restriction of the vector x to indices 
in the subclass U and W denotes the complementary set 



of U. Intuitively, this means that the drift vector-field has 
limits when we make the number of users of some of the 
classes go to infinity, and that we can interchange the order 
of the coordinates when taking these limits. We assume in the 
following that the drift vector has uniform limits, so that we 
can define the asymptotic drifts : N''^' as follows: 

S^{xu) lim Six). (7) 

Here, W corresponds to the "saturated" classes for which we 
let the number of users go to oo. We define the stochastic 
process as the Z^-dimensional stochastic process corre- 
sponding to the original process seeing an infinite number of 
users of class k eW^ and let tt^ denote its stationary measure 
assuming it exists. We define the averaged drift vectors by 

^ ^'^(x^^ix)- (8) 

Finally, following [12] we say that a vector field v is partially 
increasing if Vi{x) is increasing in xj for all j ^ i. These 
assumptions, which are crucial to prove the convergence 
towards the unique strong fluid limit, are verified for many 
cases of interest, see the next lemma. 

Lemma IV.2. A priority index policy or a BR policy with non- 
state dependent tie-breaking rule (i.e., independently of the 
numbers of users) induces a partially increasing drift vector 
fleld with uniform limits. 

Proof: We prove the lemma for BR policies, the other case 
being similar When increasing the number of users of one 
class only, the probability that this class has at least one user 
in its best possible state is increased. Hence, given that the 
tie-breaking rule does not depend on the number of users, 
the probability that this class is served is increased while the 
probability that a user of another class is served decreases. 
This implies that the drift vector field is partially increasing. 

By the independence of the channel variations, the proba- 
biUty that class i has at least one user in its best state 

is 1 — (1 — qi.Ni)^\ where Xi is the number of class-? users. 
Hence, when the numbers of class-i users, i e W^, grows large, 
the probability of having in each class in W at least one user 
in its best state (and hence causing a drift S^{xu)) converges 
to 1. Together with the property that the tie-breaking rule does 
not depend on the number of users, this implies that 

6{x) = S'^ixu) n (1 - (1 - 9..wJ"0 + (9) 

as Xk — ^ oo, k e W^, which in turn implies the uniform 
convergence of d{-) to S^{-) for any U. □ 

We now state the main result of this Section, the description of 
the strong fluid limit. The proof can be found in Appendix A. 

Tlieorem IV.3. For a given policy f inducing a partially 
increasing drift vector fleld with uniform limits, we have 

lim P( sup lY^^^s) - / (s)| > e) = 0, for all e > 0, 
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with {t) a piece-wise linear function that can be described 
recursively as follows. Let Uq ~ $ and Tq ~ 0. Then we have, 



dylit) 
dt 



Si'''', te[T/,T^, 



with tI^^ = r/ 



mm 



and Ui+i = Ui \J argmin ^~/ui ' 



(10) 
(11) 

(12) 



ith Sl''^' 

We can now give our definition of the strong fluid limit. 



and if there exists no k G lAf with Sf^'"' < 0, then T^j^-^ — oo. 



Definition 7. For a given policy f inducing a partially 
increasing drift vector field with uniform limits, we call the 
process y^ (t) (as obtained in Theorem IV.3) the strong fiuid 
limit for the policy f. 

Remark 3 (Calculation of the averaged drifts). Theorem rV.3 
characterizes the strong fluid limit as a piece-wise linear 
function with slopes 5^. In practice, the calculations of these 
slopes involve: 

• deriving the asymptotic drifts (see (7)), 

• calculating the stationary distributions of X^, 

• averaging the asymptotic drifts with these stationary 
distributions (see (8)). 

For instance, assume K — 2, Ni — 2, N2 = 1, and a Bernoulli 
arrival process. Consider the policy that gives priority to the 
best class-1 user present in the system and otherwise (i.e., 
when there is no class 1) serves a class-2 user Then 

= (Ai — /ii^Wi, A2), 

(5l^l(xi) = (Ai - Sl(xi), A2 - M2.Af2 1(2^1=0)), 
with Si{xi) = ^i,Arj(l-(l-gi^ArJ^i)+^l,Ari_l(l-(7l^ArJ^i. 

The process is a 1 -dimensional Markov chain with 

stationary distribution 



7rli}(xi) = Cn 



Ai(l-gi(j-l)) 
(l-Ai)si(j) ' 



where C is a normalization constant. The average drift can 
now be computed using (8). 

In the specific case of BR policies with a priority-type tie- 
breaking rule, we can in fact explicitly derive the strong fluid 
limit by making use of rate-conservation arguments. This will 
prove to be very useful to obtain fluid optimality statements. 

Proposition IV.4. Consider a BR policy with a priority-type 
tie-breaking rule. Let us reorder the classes according to the 
priority ordering. The averaged drift vectors are 



S ~ (Ai — A^i.Afj, A2, 
if Ti < 00, then 



,Ak), 



Al 



,Ak), 



(13) 



(14) 



and if Tfe^i < 00, then Uk-i = {1, . . . ,k — 1} and 

k-l . 

^"'•-i = (0,...,0,Afe-Mfe,A,,(l- V^),...,Ak). (15) 

Proof: Using Lemma 1V.2, the drift S{-) associated to a BR 
policy is partially increasing with uniform limits, hence the 
strong fluid limit is given by Theorem 1V.3. When U = %, 
there are infinitely many users of each class and hence there 
is always a class-1 user in its best state, which directly implies 



(13). Note that Ti < 00 if and only if 



< 1 (see definition 



of Ti in Theorem 1V.3). In this case the process IS 
ergodic. For Ti < t < T2, we can simplify the asymptotic 
drift 51^1 using the specific properties of the policy and rate 
conservation arguments: let Ai,xi be the event that class 1 is 
served and there are xi class-1 users when all the other classes 
are saturated. With a slight abuse of notation, let us denote by 
ttI^I (^i.xi ) the probability of event Ai ;^-^ under the stationary 
distribution of Xl^l. Since is ergodic, by rate stability 

we have the following rate-conservation equation, see (46), 



Al 



Ml,N"i 



Since Ui 



as before that ^ 



which gives that Y.^^ K^i^xt) = 1 
{1} (so in particular there is still an infinite amount of class-2 
users which are exclusively served when there are no class- 
1 users in their best state) class 2 receives service at rate 
A*2,jv. E.i = l^2,N, (1 - ^) Which gives (14). 

Consider now the case where U := {1, . . . ,k~l} (assuming 
< 1). Let Aj^xu be the event that 
class j eU is served and there are Xi users of class i, i gU. 
By rate-conservation arguments, see (46), we obtain 

^7r"(Aj-^„)^j- jv, = Aj-, j eU. 

Noting that the sets Aj^^u ^e disjoints, V j G W, this implies 
that Ex„'r'^(Uiew^j,:r«) = Ejgw^ITT;-- Since class k is 
only served when no class-? users are being served, i e 
U, there is a class-fc departure with probability ^k,Nk{^ — 
Ti^ {[Jj^uAj,xu))- Hence, we obtain Equation (15). □ 

Remark 4. For all BR policies where the scheduler chooses 
with probability to serve class k when a subset U of classes 
has at least one user in its best channel condition, the fluid 
limit in the interior of the orthant has a drift given by: 



(Al 



{1,...,K} \ {1---,K} 



(16) 



However, in general we cannot explicitly derive the second- 
vector flelds. An exception is the case of two classes. Then, 
using the rate-conservation argument as in the previous propo- 
sition, we obtain (assuming w.l.o.g. that class 1 empties flrst) 



j{i} 



(0,A2-(1 



Al 



)l^2,N2 



Remark 5 (Weak and strong fluid limits). Though a quite 
subtle technical point, it is worth emphasizing the concep- 
tual difference between the notion of weak fluid limits and 
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the notion of strong fluid limit, introduced in Sections IV-A 
and IV-B, respectively. Note that "weak" versus "strong" 
refers to accumulation points versus unique limit. The names 
do however not take into account the mode of convergence. 

Weak limits are a powerful tool for stability if one can 
characterize that they all vanish after a finite amount of time, 
as will be used in Theorem V.2 for the set of BR policies. 
However, in general weak limits might not capture the precise 
asymptotic behavior of the process (see [13]). On the contrary, 
when having a unique strong fluid limit, the asymptotic behav- 
ior of the scaled process is completely described, allowing for 
example to obtain the policy-dependent stability conditions for 
a large class of policies (see Theorem V.l). 

V. Stability analysis 

The derivation of the weak and strong fluid Umits in 
Section IV allows us to conclude for stochastic stability. 

The next Theorem derives the stability conditions for any 
policy having a partially increasing drift with uniform limits, 
using the strong fluid limit as obtained in Theorem IV. 3. 

Theorem V.l. A policy f inducing a partially increasing drift 
vector fleld with uniform limits is stable if t/ < oo for all I, 
where T( is given by Theorem IV.3. 

Proof: If Tf < oo for all I, the strong fluid limit described 
in Theorem IV.3 is equal to for t large enough, i.e., ¥^•'^{1) 
converges in probability to for t large enough. In addition, 
the random variable Y"/'' (t) is uniformly integrable. This can 
be seen by the fact that can be upper bounded by 

Xfe(O) plus the users that have arrived until time [rij divided 
by r, which is uniform integrable, see [15, Lemma 4.5]. 

The convergence in probability to and the uniform integra- 
bility together imply that limr_i.oo IE(Y/''^(i)) = 0, for t large 
enough, V k, see [7, Theorem 3.5]. Using an extended Foster- 
Lyapunov criterion as expressed in [18] or [28, Corollary 9.8], 
this implies the positive recurrence of X-f{-). □ 

In the following theorem we state the maximum stability 
condition, and prove that any BR policy achieves maximum 
stability. The proof is based on the weak fluid limit characteri- 
zation as given in Section IV and can be found in Appendix B. 

Theorem V.2. The maximum stability condition is 

j:—<i- (17) 

In addition, any BR policy is maximum stable. 

Condition (17) was recognized as the maximal rate stability 
condition in [1] and as the maximum stability condition under 
a time-scale separation assumption in [9]. 

We note that SB, PI and PB are stable under the maximum 
stability conditions (they belong to the class of BR policies). 
The intuition behind Theorem V.2 is that asymptotically the 
system under a BR policy behaves as a classical work- 
conserving system where class k has departure probability 
l-''k,Nk- On the contrary, other policies, including RB and the 
cfi-mle, spend (at the fluid scale) a non-negligible fraction 



of time serving users that are not in their best states, and 
are therefore not maximum stable. For an example, we refer 
to Section VII where we numerically obtain the stability 
conditions for RB and the c/i-rule, making use of Theorem V. 1 . 

Remark 6 (Overload). When J2k=i ^kl l^k,Nk > 1 system 
is said to be in overload. That is, there does not exist any policy 
that can make the system stable. Theorem TV.S is however still 
applicable, providing us with the rates at which the number of 
users in the different classes grow: given a;(0) = 0, the growth 
rate of the number of users over time is given by X-f''^{r) jr — 
Y^''^(V), which in the limit is equal to S^'^. This gives us 
a mean to compare the performance of various policies in 
overload ( see as well Sections VI and VII). 

VI. Asymptotic fluid optimality 

Besides stability, another important performance measure 
concerns the long-run average holding cost as given in (1). 
Deriving an optimal policy with respect to this criterion is 
difficult and the size of the state space makes the problem 
intractable. For this reason we introduce a related deterministic 
control problem, which allows us to prove that any BRP policy 
is asymptotically optimal for the original stochastic system. 
This emphasizes the important role of the tie-breaking rule in 
order to achieve efficient performance of the system. 

We study the following deterministic fluid control model, 
which arises from the original stochastic model by only taking 
into account the mean drifts, i.e., 

K 

miny^ Cfea;^(t), for all t > 0, subject to (18) 

k=l 

=a;fe(0) + Afci- V^fe.„ / Uk,„{v)dv, (19) 
xUt)>0, k^l,...,K, (20) 

K Nk 

^^Ufc,n(w)<l, Uk^n{v) >0, y k,n,v >0, (21) 

k=l n=l 

and the control functions Uk.n{v) being integrable. Here x'^(t) 
represents the amount of fluid in class k under control u{-). 

We remark that though in general the fluid limit of a policy 
does depend on the distributions of the random environments 
(i.e., the (jfc.n's), these do not appear in the above equations of 
the fluid control model. This is because the fluid trajectory 
Xfc(i) should be interpreted as a limit of the fluid-scaled 
process. Hence, when Xk{t) > this implies that there are 
infinitely many class-fc users so that with probability 1 there 
are class-fc users in each of the channel state conditions (this 
being independent of the exact values of the „ > O's). 

An optimal control u*{-) is derived in the following lemma. 

Lemma VI.l. Assume ci^i^n^ > C2/i2,Af2 > • • • > cxfJ^K.NK- 
The fluid control u*{-) that solves the fluid control problem is 
as follows. Let I — argminjfc ; Xk{t) > 0}. Then 

A. A- 
and ^^{t) = otherwise. 
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Proof: Let us denote ^^(t) = x'^^{t)/ Hk^^^. First, we show 
that for any feasible control u(-), we have 

j j 
J2 <* (*) ^ E "^fc (^)' fo"" alU > 0, J = 1, . . . , /C. (22) 



fc=i 



fc=i 



If X^i^i ^fe (0 = 0' '^hen (22) trivially holds. Now assume 
Y^i=i w'h'it) > 0. By definition of u*{-) this implies that 

J2k=i''^k i^) > 0' foi" all ^ ^ [0,t], since once all these 
classes empty under u*{t), they will remain empty. Since 
u*{t) gives full priority to classes 1 until j over classes 
j + 1 until K, we have that J2k=i lo '"fc n (i')dw = t. Hence, 

ELi/o<,^>)d« = i > ELii:nijo^kAv)dv > 

ELi E^r=i /o /f^"fe'»(^)d"' ^l^^'^l^ implies (22), since 

j j 

fe=i fc=i 

3 pt 3 „t 



Uk,niv)dv > 0. 



The minimization term X^feLi "^fe^-fclO '^^n be written as 



k = l 



+ {c2f^2,N2 - C3fi3,N3){w"{t) + W^{t)) + 



k=l 



K 



fc=l 



Together with (22) and Cjfij^Nj — cj+ifij+i^N.^-^ > 0, V j, we 
obtain that X^fcLi Ckf^k,Nk'u^ki^) minimized by «*(•)■ ^ 

The optimal fluid cost serves as a lower bound for the fluid- 
scaled cost of the stochastic network, see the lemma below. 

Lemma VI.2. For any policy f and for almost all sample 
paths, we have 



K 



K 



liminf Vcfer/'''(^) > Vcfexf (t), for all t > 0. (23) 



k=\ 



k=l 



Proof: Lemma IV. 1 states that for almost all sample paths oj 
it holds that lim inf ^^oo ^/'"^ (i) = ulit)- with yl{t) a 
weak fluid limit for policy / (this follows by considering 
the subsequence r; corresponding to the liminf-sequence in 
Lemma IV. 1). Note that a weak fluid limit is an admissible 
trajectory for the fluid control problem. Hence, 



K 



K 



K 



liminf ^c,.y/^'-(t) =Y^kyi{t) >Yckxi{t). 

k=\ k=\ fc=l 

This concludes the proof. □ 
Since (23) holds almost surely, it follows by Fatou's lemma 



that the lower bound holds in probability as well, i.e.. 



K 

(E 

k=\ 



CkYt{t) 



K 

E 

k=\ 



Ckxl {t) > 0) ^ 1, for an t > 0. 

(24) 

We define a policy to be asymptotically fluid optimal 
when the lower bound is obtained in probability, i.e., 
lim,,^o, P(| Ef^i CkiY^'^is) - x^'{s))\ > e) = 0, Ve > 0. 
The following Theorem characterizes a class of policies that 
is asymptotically fluid optimal. 

Theorem VI.3. Any BRP policy is asymptotically fluid opti- 
mal. 



Proof: We have 



Axl (t) _ 
dt ~ 



^k-ulj^^{t)fik,Nk, with u*{-) the 
optimal control as derived in Lemma VI. 1. This drift coincides 
with the drift of the strong fluid limit y^^^(-), see Propo- 
sition IV.4, hence y^^^{t) — x^ (t). Together with The- 
orem IV.3, we then obtain that limr-j-oo X^a^i Cfe^fe^^^''^(0 
converges in probability to '^fe^fc i-^-' any BR pohcy 

is asymptotically fluid optimal. □ 



It can be checked that the above implies that any BRP policy 
minimizes liminfr^co E(/o°° Cfci^/''^(Od^)- Unfortunately, 
this does not give any performance guarantee in terms of the 
long-run time-average holding cost as in Equation (1). Numer- 
ical experiments reported in Section VII indicate however that 
BRP policies significantly outperform all other policies. 

Note that the optimality results described in this section also 
apply in overload systems. We have the following corollary for 
the total growth rate. 

Corollary VI.4. Any BRP policy minimizes the growth rate 
of the total cost, i.e., for all e > 0, 



lim ] 

r— >-oo 



k=i Cfe^fe ' (r) T,k=i Cfe^fe 



(r) 



>-e =1. 



Proof: Combining (24) with the asymptotic optimality of a 
BRP policy, the statement is immediate. □ 

To the best of our knowledge, the only policy studied in the 
literature that belongs to BRP, and hence is both maximum 
stable and asymptotically optimal, is PI. We recall that PI 
was derived in [4] as the solution of a relaxed optimization 
problem. SB and PB will as well become asymptotically 
optimal when the myopic tie-breaking rule would be applied, 
showing the importance of the tie-breaking rule. 

Remark 7. From Theorem VI.3 we conclude that the myopic 
tie-breaking rule is crucial in order to obtain an asymptotically 
fluid optimal scheduling policy. In view of equation (2), we 
note that under this myopic rule, higher priority is given 
to users with smaller service requirements, E(Bk). Hence, 
BRP policies appropriately mix size-based information with 
achieving opportunistic gains. This is in agreement with the 
findings of [30] where the authors investigate the tradeoff 
between prioritizing small users and opportunistic scheduling: 
They show that if the opportunistic capacity is upper bounded 
and increases as 1 — a^, with a € [0, 1) and x the number 
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Channel state (n) 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


Transmission rate (kb/s) in CDMA 


38.4 


76.8 


102.6 


153.6 


204.8 


307.2 


614.4 


921.6 


1228.8 


1843.2 


2457.6 


Probabilities in CDMA 


0.00 


0.01 


0.04 


0.08 


0.15 


0.24 


0.18 


0.09 


0.12 


0.05 


0.04 










0.05 





0.23 





0.42 





0.21 





0.09 


<l2,n 








0.15 





0.33 





0.52 














Ml,n 








0.017 





0.033 





0.1 





0.2 





0.4 


M2,n 








0.017 





0.033 





0.1 















TABLE I 

Transmission rates and channel condition probabilities in the CDMA IxEV-DO wireless network, as reported in [5]. 



of users, then a significant improvement of performance can 
be achieved by exploiting information on the service time 
requirement. In our model the capacity has this behavior, 
see for instance Equation (9), and as will be observed in 
the numerical results, exploiting size-based information indeed 
allows to obtain significant improvements. 

VII. Numerical experiments 

We consider a CDMA IxEV-DO system with two classes 
of users {K — 2). Time is slotted, with the length of one 
slot being tc = 1.67ms. In each time slot, one new class-Zc 
user arrives with probability Aj.. We choose 10.257 kb as the 
expected service requirement of both a class- 1 and class-2 user 
Associated to the state of the channel, we have transmission 
rates (kb/s), see Table I (taken from [5]). We assume that 
class-1 users have five possible transmission rates while class- 
2 users have three. The corresponding probabilities {qk^n) 
are given in Table I. In addition, applying equation (2) we 
calculate the departure probabilities ifJ.k,n)- We fix A2 = 0.05, 
so A2//i2,7V2 = 0-5. We set ci — C2 — 1, so that we are 
interested in minimizing the expected total number of users 
in the system, see Equation (1). In addition, we compare the 
performance of the policies SB, RB, PI, PB and the c/i rule, 
which were introduced in Section III. 

Before presenting the numerical results we first summarize 
the main conclusions that we will make in this section: 

• We prove that not all policies obtain maximum stability. 
More precisely, we calculate the stability conditions under 
RB and the c/i-rule and observe that these are much more 
stringent than the stability condition for BR policies (e.g. 
SB, PB and PI). 

• The drifts of the fluid limit, , which are calculated 
numerically (and in some cases theoretically), provide 
very important insights on the performance. In particular, 
insightful monotonicity results in the tie-breaking rule are 
obtained with respect to the performance of the system. 
In addition, the drift analysis allows to show that PI 
outperforms all other policies in terms of the growth rates. 

• Our simulations illustrate that the tie-breaking rule has 
a very big impact on the performance of the system 
and we find that combining opportunistic scheduling 
with the myopic tie-breaking rule gives optimal mean 
performance, as was also suggested by the asymptotic 
fluid optimality result of BRP policies in Section VI. 



a) Fluid limit: We first illustrate how the scaled process 
converges to the fluid limit. We take r = 10000, y(0) = 
X{0)/r = (1,1) and plot the scaled processes Y{{t), Y{{t), 
and Y{{t) + Y2'{t) for different policies, see Figure 2. In this 
simulation we set Ai = 0.14, so \i/ij.i.Ni = 0.35. 

We describe the fluid limit y-^{t) as defined in Theorem IV.3. 
When both classes are saturated, i.e., U the drift is 

= (Ai - aVi,JVi , A2 - (1 - «-^)/i2,JvJ, (25) 

see Remark 4. Here is a random tie-breaking rule, i.e., in 
case of a tie, is the probability that class 1 is favoured over 
class 2. For our set of parameters, the best class-1 user under 
the c/i-rule and RB is always preferred over the best class- 
2 user, i.e., there occur no ties, hence one can set — \ 
in (25) for / = cfi,RB. For PI, SB and PB we do have 
ties, and we set a^^ = 1 (since PI applies the myopic tie- 
breaking rule) and a^^ — a^^ — 1/2 (since SB and PB apply 
a random tie-breaking rule). In Table II we present the so- 
obtained values for S^'^. From the drifts it is clear that under all 
policies class 1 empties before class 2. The moment that this 
happens, t/, can be derived from Theorem IV.3 and satisfies 
^ ^ j,RB ^ j,SB ^ j.PB^ ^igQ ¥igms 2 a). 

For t( < t < t/, the drift of class 1 is 0, whereas 
the drift of class 2 is going to depend on the policy. From 
Proposition IV.4 we have that for all BR policies (e.g. PI, SB 
and PB) Sf'^^"^ = (0, A2 - ^^2,^2(1 - Ai//ii,ArJ). For the c^i 
rule and RB we calculate the drift numerically using Remark 3. 
In particular we observe that these drifts are positive for the 
latter two policies, which implies instability of the system, as 
can be seen in Figure 2 b). We observe that for t < t( the 
number of class-2 users increases under policies PI, c/t and 
RB, while for SB and PB, the drift of class-2 users is negative. 

A direct consequence of the drift function is that SB, PB, 
and PI (in fact all BR policies) empty the system at the 
same time (under the maximum stability condition), i.e., t/ 
is the same (this can be seen directly from Equation (45) for 
example). However, the performance of a policy will depend 
on the order in which classes are served. In the fluid limit, 
this is fully determined by the choice of the tie-breaking rule. 
Note that, as can be seen from Figure 2 c), PI, (and hence any 
BR policy with the myopic tie-breaking rule) minimizes the 
total number of users at any moment in time. 

b) Stability region: We now vary the value of Ai from 
0.004 to 0.196, and as a consequence we have that p := 
^1/ l^i,Ni + A2//i2,JV2 varies from 0.51 to 0.99. The policies 
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Fig. 2. (a) Scaled number of class- 1 users, (b) Scaled number of class-2 users, (c) Scaled total number of users 




Fig. 3. (a) Mean number of users and stability thresholds, (b) PI under different tie-breaking rules: relative degradation (in %) over PI with a = 1, (c) 
Scaled total number of users in overload, p = 1.1. 



f 


Class 1 


Class 2 


Class 1 


Class 2 


PI 


-0.26 


0.05 





-0.015 


c/i-rule 


-0.26 


0.05 





0.0096 


PB/SB 


-0.06 








-0.015 


RB 


-0.26 


0.05 





0.0004 






TABLE II 








Drift of the fluid limit. 










f 


Class 1 


Class 2 


Class 1 


Class 2 


PI 


-0.16 


0.05 





0.01 


c/^-rule 


-0.16 


0.05 





0.036 


PB/SB 


0.04 









RB 


-0.16 


0.05 





0.029 



TABLE III 

Drift of the fluid limit in overload system. 



PI, PB and SB belong to the BR policies, and are hence stable 
when p < 1. For the c/i-rule and RB the stability condition 
can be calculated (numerically) by setting 52^^^ equal to zero 
and following the steps described in Remark 3. In particular, 
the c/x-rule is stable if and only if p < 0.79 and RB is stable if 
and only if p < 0.84. In Figure 3 a) we plot the mean number 
of users for different values of p and we observe that the mean 
number of users (and hence the mean delay) for these policies 
grows to infinity as the load approaches the critical value. 

c) Impact of Tie-Breaking rule: We study the impact 
of the tie-breaking rule on the performance of the system. 



In order to investigate this issue in more depth, we simulate 
PI under different random tie-breaking rules, i.e., we let the 
probability a vary from until 1 (Recall that the parameter 
a is the probability, in case of a tie, that class 1 is favoured 
over class 2). We emphasize that PI as defined in [4] uses 
by default the myopic tie-breaking rule, i.e., a^^ — 1. In 
Figure 3 b) we plot the relative degradation (in terms of the 
mean number of users) over PI as we vary a. We observe that 
the degradation in the mean performance is decreasing as a 
increases. In the fluid-scaling system a similar observation can 
be made: from (25) and Remark 4 it follows that the drift of the 
total number of users of the strong fluid limit (as obtained in 
Theorem IV. 3) is decreasing (or constant) in a, for all time t. 

The results show that the myopic tie-breaking rule, which 
was proven to be asymptotically optimal in the fluid limit (see 
Theorem VI. 3), is in practice indeed optimal when minimizing 
the mean number of users. In addition, the relative degradation 
of the tie-breaking rule with a = 1/2 (compared to the myopic 
tie-breaking rule a^^ — 1) can be very large. For example, 
for p = 0.8 the degradation is 29% and for p = 0.9 it is 45%. 

d) Overload: In Figure 3 c) we plot a trajectory of the 
total scaled number of users Y{{t) + Y{{t) (for r = 100) 
when Ai = 0.240 and p = 1.1, so all policies are unstable. In 
Table III we give the values for the drifts (growth rates) of the 
fluid limit (t) in this overload setting. In this example, the 
worst performance is under SB and the best performance is for 
PI. This in contrary to the stable regime where SB is maximum 
stable with a performance strictly better than the c/i-rule and 
RB, see Figure 3 a). This implies that the performance of 
this policy can differ very much between stable and overload 
regimes. In addition, we note that the total growth rate of the 
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system is always minimized under any BRP policy, that is a 
best-rate policy with the myopic tie-breaking rule, as is proved 
in Corollary VI.4. 

VIII. Conclusion 

We have characterized the classes of policies that are 
maximum stable and asymptotically optimal in a system with 
random environment. An important conclusion, validated by 
numerical experiments, is that the tie-breaking rule has a 
tremendous impact on the performance. Our analysis also 
shows that simple priority-index policies like PI or SB with 
a tie-breaking rule, are stable and asymptotically opti- 
mal. While in this model we assumed geometric service 
requirements, we do believe that direct extensions of all our 
results exist for phase-type distributed service requirements. 
In particular, we expect that an optimal tie-breaking rule will 
be of a simple priority-index type. 

Acknowledgements We are grateful to S.C. Borst (TU/e and 
Bell-Labs) and P. Jacko (BCAM) for many fruitful discussions. 
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Appendix A: Proof of Theorem IV. 3 
We first derive a property on the drift vector. 

Lemma VIII.l. Suppose 5^ {■) has uniform limits. Then for 
any lA and any compact set C C M^^^^' there exists an L > 
and Cr with limr-->.oo Sr — such that 

\Sf{x)-S^^fixu)\<er + L\^-y\, 

Va; e , xu^ e 1^1^+' , y G C c R^+l^ . (26) 

Proof: Consider e > 0, and y £ C such that Vi > e, Vi G W^. 
Define Bry.re C the ball of center ry and radius re. 

Assume that the drift function d{-) has uniform limits. This 
implies that for all xn the sequence 

= sup sup (x) - S'^'^{xu)\, 

is converging to 0, when r ^ oo. Hence, remarking that 
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I Bdf. 



y\> e when xu<: ^ Bry^rt, we obtain that 



\6f{x)~5'''f{xu)\ < sup \6f{x)-S'''f{xu)\ + 
+ sup \Sf{x)-S'''f{xu)\ 

< er + L\^-yl 
r 

n r\ sup ,. ctK (x) — &^ (xu ) I ^ 

for i 2^ > Px»cgB.„.„ I ^ ; \_un ^^^^^ ^ _ 

e — e ' 

niaxfc,„(Afc,/Xfc,„). □ 

In what follows, we will use the martingale decomposition of 
the process (see [35]), i.e., we define M(-) such that 

t-i 

X{t)=X(Q)+Y,5{X{s)) + M(t), (27) 

where M is a martingale, and |A'/(0)| = 0. The following 
lemma gives a useful property for the martingale M{t). 



Lemma VIII.2. It holds that E (^|A4(i)|-'j < C't, Vfc = 
\ . . . ,K, for some constant C > 0. 

Proof: Using that the drift is bounded and the arrival incre- 
ments have a finite variance, it immediately follows from (27) 
that E (^lMfc(i)l^) < oo, \/t. Hence, 



E{\Ahit)\- 
<E(\Mk{ 



(28) 



^e(|(M,(s)-A4(s-1))|' 



see [35, Chapter 12.1]. From (27), we have that for all s, 

\Mk{t) ^ Mkit - l)\ 

< \Xk{t) ^ Xkit - l)\ + \S{X,{t ^ 

Since the drift is bounded, and the arrival process has finite 
mean and variance, this implies that for C" > a constant 

E(|Mfe(i)-A4(t-l)|2) <C'. 

Thus, the statement follows from (28). □ 

We prove Theorem IV.3 for two classes of users, the general 
case being notationally much more cumbersome but mathe- 
matically equivalent. The trajectory of the fluid limit is defined 
piece-wise. We refer as the first part of the fluid limit to 
the case t < Ti, and as second part of the fluid limit to 
the case Ti < T2, and we shall prove the convergence 
separately for the two parts. The basic techniques used to 
prove the convergence of the first part of the trajectory are 
fairly standard and we proceed using the tools of [20]. Let 
us however emphasize that the proof relies crucially on the 
extended Lipschitz condition (26). Verifying this condition 
might be difficult in practice. In Lemma VIII. 1, combined 
with Lemma IV.2, we however showed that the conditions are 
satisfied for various scheduling policies of interest. Proving 
the convergence of the second part of the trajectory is much 
more subtle as it involves an averaging phenomena: one of 
the classes reaches a stationary regime while the other class 



keeps being macroscopically big. We must therefore rely on 
stochastic comparisons and on the ergodic theorem for Markov 
chains. Such phenomena have been observed and analyzed 
in [18] for homogeneous random walks in the positive orthant. 

First part of the trajectory: Assume strictly positive initial 
conditions for both classes, i.e., a;i(0) > and 2:2(0) > 0. 
At time t = all classes are saturated, hence in the first part 
of the trajectory, t < Ti, we need to consider U — Uq = 
0. Consider the deterministic function y{-) as defined in the 
theorem, y{t) = a;(0) + 6^t, and we define an error function 



e'-(i) := E( sup {s) - y{s)\ ] . 

\0<s<t 



We obtain from (27) that 

LrtJ-l ^ 



\Y'-it)-y{t)\<l kx'-{s))-6' 



s=0 



+ 


M'-(LrtJ) 




r 



(29) 

We first control the martingale. Using successively Cauchy- 
Schwartz, Doob's inequality and Lemma VIII.2, we get 




< 4E 



We now control the drift part in Equation (29). Since the drift 
has uniform limits, we obtain from Lemma VIII. 1 that 

\S{X-{s))-S^\<er + L\^^-y{-)l y s < rt. 

r r 

Hence, bounding the Riemann sum for piece-wise constant 
functions by an integral, there exists a k > such that 



r 

s=0 



< €rt + K 



sup 

0<s<s 



X''{rS) 



y(s)|ds 
(31) 



Taking expectations on both sides we obtain 



E 



< Ert + K 



Together with (29), (30) and (32), we deduce that 



e'^(s)dsj . 

(32) 



Ar(LrsJ) 




E ( sup 

,s<t 



<ert + K e'^{s)dsj + 
Hence, for any e > there exists rg such that, for a C" > 0, 

e''{t) < C" (^t + ^ e''(s)ds^ , for r > tq. 
Using Gronwall's lemma [27], we obtain that for any r > vq, 
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for all t < Ti. Hence, limr_>oo e''(<) = for t < Ti. Since 
e^{t) = E(supo<,<, \Y-is) - y{s)\) > P(supo<,<, lY^is) ~ 
y{s)\ > e') • e', we obtain limr_j.oo IP(supg<s<j |F''(s) — 
y{s)\ > e') = 0, for t < Ti. 

In case Ti — oo, the Theorem IV.3 is now proved Vi > 0. In 
the remainder of the proof we therefore assume that Ti < oo. 

Second part of the trajectory (one stationary class): We 
decomposed the trajectory of y into a first part (t < Ti) and 
a second part (t > Ti). By the Markov property, we can study 
the second part of the trajectory supposing that Y'^{Ti) = 
{xi,y2{Ti)r). Since Ti < oo, the process X^^^ is ergodic. 
Using the monotonicity of the drift, we have that X^t) <st 
Xl^'^(t) fort > Ti (this can be observed by coupling the two 
processes and noting that in the case of an infinite number of 
class-2 users, the service rate for class 1 is the lowest possible 
since there will always be a class-2 user in its best state). This 
implies that the family {X[{t))r is tight and that ^• 

0, as r — )■ oo, for t > Ti. At time Ti class 2 is saturated, 

1. e., Ui = {1}. In what follows we assume that Ti < t < 
T2. Consider the deterministic function 2/2 (•) as defined in the 
theorem, y2(i) = ^2(^1) + 6^ {t - Ti), Ti < t < T2, and 
define the error function 

E^it):^ sup \Y2%s) - y2{s)\. 

Ti<s<t 



assumption) we obtain that 



From (27) we obtain 

Using the triangular inequality we obtain, 

LrtJ-i 



(33) 



Em<l E \52iX^is))-5P{Xlis)) 

s=[rTt] 
lrt\~l 



(34) 



= LrTiJ 







+ sup 


0<s<t 


r 



From Lemma VIII. 1 we obtain that there exists with 
limr_>.oo £r = 0, such that Vs = L^TiJ , • • ■ , L^^J ■ 

MX^-is)) - 6P{Xl{s))\ <er + L\^ - y2{-)\. (35) 

r r 

We will now derive a bound on |4^^(X[(s)) - ~5\^^\. In 
order to do that, we define a one-dimensional process Xi 
which has the same dynamics as our initial process but for 
which we fix the class-2 users to be equal to r{y2{Ti) — 9), 
with Q < < 2/2(21). Let tt^'*" be the stationary distribution 
of Xi. Adapting Lemma 6 in [12] to our discrete time setting, 
we obtain that 



lim 7r^'''(x) = 7r{i>(a;),Va;. 



(36) 



We define T"^ = inf{t : X^(rt) < r{y2{Ti) - 9)}. Using 

coupling arguments, we see that T"^ > Ti almost surely, while 

'{1}/ 



for Ti < t < Xlit) <st Xlit) <,t Xl''{t). Together 
^2 



with the fact that 62^^-) is increasing (using the monotonicity 



< 



s=lrTi\ 

^ LrtJ-i 

s=LrTiJ 



4rTiJ 



6l'Hx[{s)) 



Sl^\xl^\s)) for t < T"- . (37) 



Since 62^^ is uniformly upper-bounded, we have that 



lim E^'"'(^)'^2 ^( 



E 



lim 7r^'''(a;)4^^(a;) 



Hence, we obtain that for all e > there exists such that 
^^>p7r^'''°(a;)(52^^(a::) > Jj^^ — e, for all r > tq, and applying 
the ergodic theorem for Markov chains we get 

L'-tJ-l 

lim - V ^(s)) = (^-^l)^^^ a.s. and 

\rt\-\ 

lim - E 4'^(XUs))>(t-ri)(4^>-e), a.s.. 

r— foo T ^ — ^ 

s=rTi 

Together with (37) we now obtain that there exists an tq such 
that for all r > tq 



4^^(^r(s)) -^^^^1 < e,a.s., for t < T\ (38) 



s=LtTiJ 

Let us now condition on the set of events ^\{e) — {lo : 
sup^<t '^'^^''^j:''''^^! < e}. From (34), (35) and (38), and using 
an inequality similar to (31), we obtain that for any sample 
path in ^\{e) (with e small enough) it holds that 

£^2 (i) < C"{et + / El{s)ds), for all t<T\ (39) 
Jo 

with r large enough, and C" > 0. Using Gronwall's lemma 
[27], equation (39) implies that E^{t) < C"et{l+te*). Hence, 
we can conclude that for a given e > 0, there is an e > such 
that for each sample path in i^\{e) it holds that 

E'^{t) < e, for all t < and r large enough. (40) 

We define T°° := - ^ if 5^ < 0, and = 00 

otherwise. Assume uj e ^l{e). If T"" < < 00, then 



m - 



so T'' > T° 



see also Figure 4. When T°° — 00, it is easy to check that 
= 00 for e small enough: Consider an e < 6*. If < 00, 
then 1^2'' (7"^) "2/2(^01 > ^- However, since lu e n[{e) we 
also have \Y2{T^) — y2{T'^)\ < e, which gives a contradiction 
with the above. See also Figure 5. 

Therefore, given t such that t < T°°, there exists an e 
small enough such that t < for all r large enough. Hence, 
from (40) we obtain for t < T°°, 



sup \Y2%s) - y2is)\ > e 

Ti<s<t 

PiEUt)>e)<¥{n/niii)). (41) 
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!/2(Ti) 



y2(T,)-6 



Fig. 4. Case T°° < oo. 




2/2 (Ti 



y2{Ti)-e 



Fig. 5. Case r°° = oo. 




From (30), we obtain lim,.^ooE ^sup^<f 
Since E 



M'-(L-rsJ) 



> P(sup, 



we obtain that = P(sup 



s<t 



M'-(Lr^J) 



M''([rsJ) 
r 

r 



= 0. 

> e1 < 



0, as r — > oo. Together with (41), 



this implies that for t <T° 



lim ] 

r— f CO 



sup \Y^{s)-y2{s)\>e] = 0. 

Ti<s<t 



y2(t) _ 



Finally, letting 6* f V2{Ti), and hence r°° f Ti- 

^2 

we obtain that (•) converges in probability on compact sets 
to 2/2(-) in the set [Ti,r2). □ 



Appendix B: Proof of Theorem V.2 

If condition (17) is not satisfied, the system is not rate 
stable, see Lemma V111.3 below, which precludes stability. 
Hence, (17) is a necessary condition for stability. We are 
now left with proving that BR policies are stable under 
condition (17). In order to do that, we will take the following 
steps: first, we prove that, under any BR policy, the set of 
sample paths where service is given to a user that is not in 
its best state is asymptotically almost surely empty. Then we 
prove that the fluid limit will be equal to zero almost surely 
for some time T large enough, if condition (17) holds. This 
will then allow to conclude the proof. 

In the remainder of the proof we focus on BR policies and 



assume condition (17) to be satisfied. Let us define 



T := 



y-if vim 



1 



< OO. 



(42) 



We define the random variable T[ — inf {i : J2^=i ]ir^ — ^} 



for < e < 1 and let = min {T^,T}. Consider the event 
Ar which occurs if, during the time interval [0, [rT^J], there is 
at least one user which has been served while not being in its 
best possible state. Since any BR policy gives absolute priority 
to users which are in their best possible state, it follows that 



lrT:\ 



K 



s=0 



k=l 



Since <T and using basic algebra, it is easy to check that 
there exists a constant ^ e [0, 1) such that 



HAr) < 1 - (1 - n 



r\rT 



We will now show that 



0. 



(43) 



(44) 



In order to prove this, we need < Noting that 

log(l + x) = X + o{x) when x is close to zero, and applying 
this fact to log((l - CY^), we obtain that (1 - C")'^ = 
e"''"^^ for large values of r. Using Taylor expansions 

and the fact that < it follows that. 



r=l 



r\rT\ 



oo 

El 

r=l 



Hence, we can apply Borel-Cantelli's lemma [19] and we 
obtain Equation (44). 

We will now show that at time T the weak fluid limit is 
almost surely equal to zero. From Equation (44) we obtain 
that, almost surely, event occurs when r is large enough. 
Hence, only users in their best possible state are served in the 
interval [0, j'T^] when r is large enough, i.e., Tj^^'^it) — 0, 
for n 7^ Nk and t < rT^, for r large enough, a.s.. Hence, 
for almost all u we have that in the weak fluid limit presen- 
tation of Lemma IV. 1 it holds that EfcLi'''^Wfc(0 ~ ^ 
T^J^{t) = for an n ^ iVfc, t < liminf^^oo T^- Therefore, 
for t < liminfr-i-oo we have that any weak fluid limit 
y^^{-) satisfies 



fe=l 




Let Tp < oo denote the moment that y^i._i — — — = e. For a 
given sample path w, let be the subsequence correspond- 
ing to lim inf r-i-oo . By Lemma IV. 1 we know that there 



exists a subsequence rfc, of such that | ^ ^' — - — 

T.k - — I ^ e'' for > and / large enough. Hence, 

if T"^"' < r„ we have that - < , ^Z, , i.e., 

T/' > - i-^A^/z^few • ^^'^^ linie'^oliminf^^oor^ > 
Te, the description (45) holds for all t < T^. Now letting e 
go to zero, then — >■ T, hence Equation (45) holds for any 
t < T, and in particular (T)\ = 0. 

So we conclude that for almost all sample paths, any 
weak fluid limit converges to zero at time T, in particular 
the sequence corresponding to the liminf and the limsup. 
Hence, one can conclude that lim,.^oo Y^'"' {T) = yl{T) = 0, 
k = 1, . . . ,K, almost surely. Since Y^'^ [T) converges almost 
surely (and therefore, also in probability) to 0, we can use the 
same argument as in Theorem V.l, in order to conclude that 
the system will be stable under policy /, which finishes the 
proof of Theorem V.2. 

Below we state the rate stability result. 

Lemma VIII.3. Assume X[t) is rate stable, i.e., 
limt^eo X,{t)/t ^Ofar all i. Then J2k=i ^ < 1- 

Proof: We first derive rate conservation equations for the (per 
class) number of users. Using the martingale decomposition 
of the Markov chain, we obtain that Xi{t)/t = Xi{0)/t + 
Y^*s=i ^i{Xis))/t + Mi{t)/t, where Mi is a square martingale 
such that E[Mi(t)'^] < Ct, with C > a constant. Hence, 
^ ^ 0, almost surely. So, if ^ then 

1 * 

lim -V =0. (46) 

t-i-oo t ^ — ' 
s=l 

Hence the time average of the Cesaro mean of the service Si{-) 
dedicated to class i is equal to A;. Given that we consider a 
system with capacity 1, X^ili — 1'^^' which, combined 
with the previous equation, gives that J2iLi ^' < 1- ^ 



